Integration of multiple feature sets for reducing ambiguity in automatic speech recognition

This thesis presents a method to investigate the extent to which articulatory based acoustic features can be exploited to reduce ambiguity in automatic speech recognition search. The method proposed is based on a lattice re-scoring paradigm implemented to integrate articulatory based features into automatic speech recognition systems. Time delay neural networks are trained as feature detectors to generate feature streams over which hidden Markov models (HMMs) are defined. These articulatory based HMMs are combined with HMMs defined over spectral energy based Mel frequency cepstrum coefficient (MFCC) acoustic features through a sequential lattice re-scoring procedure. The optimum phone strings are found by maximizing the log-linear combination of acoustic and language models likelihoods during recognition. The associated log-linear weights are estimated using a discriminative model combination approach. All the experiments are performed using the DARPA TIMIT speech database and the results are presented in terms of phone accuracies.